perm filename DGPROP.TXT[COM,LSP] blob
sn#787321 filedate 1985-02-22 generic text, type T, neo UTF8
From: Dan Oldman (Data General)
To: cl-foriegn-function-call
I am sorry for the delay in becoming active on this committee.
I still don't have an ARPANET account. If you want to reach
me we have to resort to more traditional media. My address
is:
Dan Oldman
Data General Corp.
62 T.W. Alexander Drive
Research Triangle Park
North Carolina 27709
Phone: (919) 549-8421
I'll accept collect calls.
Attached is an alternate proposal for foreign communication that
is taken from Data General's Common LISP implementation. It has
three significant components:
1. Foreign Structures - which are similar to Spice's
Alien Structures, but different in that they depend on the
ordering and type of components to define the layout of
the structure instead of the programmer stating the
exact positions. It also contains a facility (the binder
function) that allows the structures to share space with
external static data of other programming languages.
2. Foreign Entries - which are similar to VAX LISP's
DEFINE-EXTERNAL-ROUTINE.
3. Representation types - a facility that the other two
components use to encapsulate information about
representation and passing rules of the data types of
different languages. For example, on DG machines, a C
integer is 32 bits passed by value, while a Pascal integer
is 32 bits passed by reference.
You will probably note that this facility has a number of
parameters that are specific to the Data General MV Architecture
and its Common Language Calling Conventions. This is unavoidable
since this facility has to be able to describe all the passing and
representation rules assumed by the foreign routines. I would not
expect these parameters to become part of the standard. Every
implementation will be obligated to provide its own parameters and
values.
It is my hope that we will be able to encapsulate these architecture
specific rules for an interesting subset of DG languages into
pre-defined representation types. This will eliminate the need
of the programmer to know how arguments are passed.
This proposal is not complete. In particular it still needs a story
for arrays and union types which I have in mind but have not
specified.
Chapter 27
Foreign Calling
The DG COMMON LISP system provides facilities for calling
routines written in languages other than LISP. These routines and
any data that they use are called "foreign" because they do not
follow the discipline required by LISP storage management. Support
for foreign code falls into four distinct but interdependent
categories:
* Representation types. A LISP type specifier, such as string
or integer, characterizes a class of objects whose actual
representation is hidden from the user. Within the LISP
domain, the programmer does not need to know how LISP's
data abstractions are implemented. If a LISP string or
integer is to be passed as an argument to a foreign
routine, however, it must be represented in the form that
the foreign language will accept. The PASCAL language, for
example, expects a character string to be prefixed by a
16-bit integer indicating its length. If both LISP and
PASCAL are to access the same string, then a mediator must
translate between the LISP object and its foreign
counterpart. The representation type performs this role.
A representation type specifier defines a bit-
representation for some class of LISP objects. An element
of the representation type accepts and returns LISP values,
which are automatically translated to and from the speci-
fied representation. The predefined boolean representation
type, for example, characterizes an object in which a LISP
value, nil or non-nil, will be stored as a bit value 0 or 1
(the typical Boolean representation of other languages).
Conversely, when an object of this type is accessed, it
yields the LISP value nil or t.
A representation type determines not only the bit
representation of its elements, but other attributes of the
type as well: for example, the rules for addressing,
passing, and returning an instance of the type.
* Foreign structures. Any object that is to be accessed by
LISP and by foreign routines must be defined as a foreign
structure. Like defstruct, the foreign structure facility
generates the routines that create and access objects
shared by LISP and foreign code. Each field of the foreign
structure is interpreted according to its specified repre-
sentation type.
A LISP object persists as long as it is accessible
from some pointer; pointers are monitored by LISP storage
management. The LISP garbage collector is free to move
objects in memory because it knows how to modify any
pointers to those objects. Foreign objects do not conform
to this discipline: the lifetime of an object and the
locations of any pointers to it are under programmer
control. LISP storage management ensures that all data
defined as foreign structures are allocated in space that
is not subject to garbage collection.
* Foreign entries. A foreign entry is a mechanism for calling
a foreign routine from LISP. The user supplies the name of
a procedure or function written in another language, and
the representation types of its arguments. The foreign
entry facility generates a LISP routine to call the foreign
routine using the appropriate protocol for passing
arguments.
* Foreign code regions. The foreign region facility builds a
directory that contains a linked image of foreign object
files.
27.1 Representation Types
-------------- -----
A representation type defines a mapping from a COMMON LISP
data abstraction to its bit representation. More specifically, the
representation type defines a container of a precise size and shape
and prescribes methods of passing the container between LISP and
foreign code. When a LISP value is stored in the container, it is
translated into the appropriate representation; when accessed, the
container yields the LISP object that is the translation of its
contents.
Like COMMON LISP data types, representation types may be
symbols or lists. When the representation type specifier is a
list, its car is a symbol, and the rest of the list holds subsi-
diary information about the representation. A symbol and a list of
the symbol, for example boolean and (boolean), are equivalent type
specifiers.
27.1.1 Predefined Representation Types
---------- -------------- -----
The following are representation types that are predefined in
DG COMMON LISP. They represent most of the data types commonly
used by other programming languages.
(signed-integer size)
Denotes a size-bit object interpreted as a signed integer. Acces-
sing such an object yields a value whose LISP type is (signed-byte
size). It is an error if size is not in the range of 1 to 32.
integer
This is an abbreviation for (signed-integer 32).
integer16
This is an abbreviation for (signed-integer 16).
(a1-signed-integer size)
This is an abbreviation for (alignment 1 (signed-integer size)).
See "Representation Type Attributes," below.
(unsigned-integer size)
Denotes a size-bit object interpreted as an unsigned integer.
Accessing such an object yields a value whose LISP type is
(unsigned-byte size). size must have a value between 1 and 32.
(a1-unsigned-integer size)
This is an abbreviation for (alignment 1 (unsigned-integer size)).
float
Denotes a LISP type single-float represented as a 32-bit normalized
floating-point number. LISP single-float numbers represent a
smaller range of values than do MV architectural single-precision
numbers. Conversion will have the same effect as
(coerce (the double-float x) single-float).
double-float
Denotes a LISP type (double-float) represented as a 64-bit normal-
ized floating-point number.
(boolean size)
Denotes a size-bit unsigned integer that holds true and false
values. When translating from a LISP object to a bit
representation, boolean maps nil to 0 and non-nil to 1. When
translating from a bit representation to a LISP object, a boolean
maps 0 to nil and non-zero to t. The size field is optional and
defaults to 1 bit; thus boolean is the same as (boolean 1).
(ascii-sequence length)
Denotes a LISP type (string length) represented as a sequence of
8-bit ASCII characters length bytes long. Any object of this type
accepts only strings of length characters.
(null-term-ascii-sequence length)
Denotes a LISP type string of no more than length characters,
represented as a sequence of length 8-bit ASCII characters. A
string of fewer than length characters is terminated by the ASCII
null character (zero). This is the normal "char" data type for the
C programming language, and the character string used by the AOS/VS
operating system.
(string length)
Denotes a LISP type (string length) represented as a sequence of
length 8-bit ASCII characters prefixed by a 16-bit integer that
indicates the number of characters in an instance. length must
have a value between 0 and 32767. This is the normal character
string for PASCAL and PL/I.
(string32 length)
This is the same as the representation type (string length), but
the prefix to the character sequence is a 32-bit integer. length
must have a value between 0 and (1- (expt 2 31)).
(bit-sequence size)
Denotes a LISP type (bit-vector size) represented as a sequence of
size bits.
(bit-set size sequence)
Denotes a LISP type (member sequence) represented as a set of size
bits. The bit-set type maps sequence onto the bit set; there must
be an element in sequence for every bit in the bit-vector. Tran-
slating from the bit sequence to LISP yields a list. A setf of an
object of this type accepts a list, and sets and clears the asso-
ciated bits.
(selection size sequence)
Denotes a LISP type (member sequence) where size designates the
number of bits in an unsigned integer. The unsigned integer
indexes the sequence. For example, an object of type (selection 8
'(a b c)) is represented as an 8-bit unsigned integer; on access to
this object, a value of 0 maps to a, 2 maps to c, and 3 is an
error. A setf of such an object maps the symbol to the associated
value.
(pointer fstruct)
This represents a 32-bit MV address; fstruct is a symbol that names
a foreign-structure type. See "Foreign Structures," later in this
chapter.
* Accessing an object of this type yields an alias of the
fstruct instance that is referenced by the pointer. If the
pointer has a value of 0 (that is, an MV architectural nil
pointer), it yields the LISP nil value.
* The setf operation on such an object accepts an instance of
the type fstruct and stores its address. When given a
value of nil, setf stores an address of 0.
* If fstruct has a :word-aligned addr-mode, this type repre-
sents a word address; if it has a :byte-aligned addr-mode,
this type represents a byte address.
The fstruct symbol that appears in a pointer representation
type need not be defined as a foreign-structure type when the
representation type is specified. This allows the user to define
structures that point to other instances of the same type. For
example:
(def-foreign-struct name-tree
(name (string 32))
(left (pointer name-tree))
(right (pointer name-tree)))
defines a structure type used to build a binary tree in which each
node holds a string and pointers to its sons.
any-pointer
This represents a 32-bit MV address. any-pointer works like
(pointer fstruct), but an object of this type need not point to any
one type of structure. A setf of such an object accepts any
foreign structure. When accessed, the object yields a foreign
structure that has no accessing functions; this structure may be
passed to a foreign-structure binder function to take on a specific
type.
(foreign-structure fstruct)
Denotes a foreign structure of the type fstruct. Use this construct
to describe structures embedded in foreign structures and to
describe foreign structure arguments in foreign entry declarations.
When you access a field of a foreign structure that has been
declared with this representation type, you get a new foreign
structure instance that is a copy of that instance. setf accepts
instances of type fstruct and copies that structure into the
appropriate place in the destination structure.
Use foreign-structure in a foreign entry declaration to pass a
foreign structure. If the structure's passing-mode is
:by-reference, then the structure's address is passed to the
function. If its passing-mode is :by-value-rj, then the structure
is copied during the call.
(any-foreign-structure size)
This is like (foreign-structure fstruct), but no specific structure
type is involved. setf accepts any structure that is size-bits
long; size must be an integer greater than zero. When accessed, an
object of this type yields a structure that has no internal
structure. Use a binder function to give this structure a specific
type.
An argument in a def-foreign-entry may be represented as
any-foreign-structure, with no size specified. In this case, the
foreign entry accepts a foreign structure of any size as argument.
27.1.2 Representation Type Attributes
-------------- ---- ----------
Four attributes are associated with each representation type.
These attributes determine the rules for addressing, passing as
arguments, and returning objects of the given type.
addr-mode
This attribute determines the kind of pointer used when passing an
instance by reference to a foreign routine. The legal values are
:word-aligned and :byte-aligned.
alignment
This attribute determines the position of an instance within a
foreign structure.
(def-foreign-struct foo
(test boolean)
(value integer))
and
(def-foreign-struct foo
(test boolean)
(pad (bit-sequence 15))
(value integer))
Both of these examples define a foreign structure 48 bits in size.
Although the boolean field occupies only one bit, the integer field
has a default alignment of 16. To define a 33-bit record with the
same fields, you may specify a bit-alignment for the integer:
(def-foreign-struct foo
(test boolean)
(value (alignment 1 integer)))
The legal alignment values are members of the list (1 2 4 8 16 32).
passing-mode
This attribute determines how to pass an instance as an argument to
a foreign routine. The legal values are :by-value-rj,
:by-value-lj, and :by-reference. An object with a :by-value
passing-mode is pushed directly on the stack; if its size is not a
multiple of 32, a pad is inserted to produce a field whose size is
a multiple of 32. The value is right- or left-justified in this
field according to the suffix -rj or -lj. A :by-reference type is
passed by pushing a pointer to the instance on the stack. If the
type has a :word-aligned addr-mode, the pointer is a word pointer;
if the addr-mode is :byte-aligned, the pointer is a byte pointer.
returning-mode
This attribute determines how foreign functions return values to
LISP:
:in-ac0-rj The result of the function is in general
:in-ac0-lj accumulator 0. If the size of the result
is less than 32 bits, it is right- or
left-justified according to the suffix -rj
or -lj.
:in-lower-16-of-ac0-rj The result of the function is in the lower
:in-lower-16-of-ac0-lj 16 bits of general accumulator 0. If less
than 16 bits in size, it is right- or
left-justified according to the suffix -rj
or -lj.
:in-ac2 A holder for the result is allocated before
the call. The word address of that holder
is passed to the foreign function in
general accumulator 2.
:in-fpac0 The result is in the single-precision
portion of floating-point accumulator 0.
:in-fpac0-double The result occupies all 64 bits of
floating-point accumulator 0.
:in-arg1-word-aligned A holder for the result is allocated before
:in-arg1-byte-aligned the call. Its address is passed in the
first argument position. A word- or
byte-pointer is passed according to the
suffix -word-aligned or -byte-aligned.
27.1.3 Default Attributes of Representation Types
------- ---------- -- -------------- -----
All of the predefined representation types have default values
for the four attributes defined above. The default passing-mode
for all types is :by-reference. Other attributes are listed in
Table 27-1.
------------------------------------------------------------------
Table 27-1: Default Attributes of Predefined Representation Types
Rep type addr-mode returning-mode alignment
signed-integer :word-aligned :in-ac0 16
unsigned-integer :word-aligned :in-ac0 16
float :word-aligned :in-fpac0 16
double-float :word-aligned :in-fpac0 16
boolean :word-aligned * 1
ascii-sequence :byte-aligned :in-ac2 8
null-term-ascii- :byte-aligned :in-ac2 8
sequence
string :word-aligned :in-ac2 16
string32 :word-aligned :in-ac2 16
bit-sequence :word-aligned * 1
bit-set :word-aligned * 1
selection :word-aligned :in-ac0 16
pointer :word-aligned :in-ac0 16
any-pointer :word-aligned :in-ac0 16
foreign-structure :word-aligned :in-ac2 16
any-foreign- :word-aligned :in-ac2 16
structure
* The returning-mode is determined by the size of the instance:
Size in Bits returning-mode
1-16 :in-lower-16-of-ac0-lj
17-32 :in-ac0-lj
33- :in-ac2
←←←←←←←←←←←←←←←←←←←←←←←←←←←←←←←←←←←←←←←←←←←←←←←←←←←←←←←←←←←←←←←←←←
To override the default attributes of any representation type,
create a new type by enclosing the existing type in a form that
specifies the new attribute:
(attribute new-value rep-type)
For example, this is the correct representation type for the C
language's int type:
(passing-mode :by-value-rj integer)
27.2 Defining New Representation Types
-------- --- -------------- -----
The predefined representation types, with possible attribute
modifications, should serve the requirements of most foreign code
applications. However, the need for new representation type
specifiers may arise. The new representation type could be no more
than a convenient label for an existing type, or it could represent
a translation of the values of an existing type.
The def-rep-type macro is analogous to the COMMON LISP deftype
macro:
def-rep-type name lambda-list {doc-string} {form}* [Macro]
The name is a symbol that identifies a new representation type;
lambda-list is a lambda-list; and forms constitute the body of the
expander function (see deftype).
If the optional documentation string doc-string is present,
then it is attached to the name as a documentation string of the
type type; see documentation.
In a typical application, def-rep-type packages an existing
type under a simple name:
(def-rep-type c-int ()
"C language int type."
`(passing-mode :by-value-rj integer))
After executing this def-rep-type, the user can denote the C
language's int data type as type c-int in place of the longer type
specifier.
To define a translator for values of an existing type, the
def-rep-type must return a translator-rep-type. A
translator-rep-type is a representation type descriptor that takes
four keyword arguments:
:lisp-type The type of LISP object that will be accepted by
setf and returned on access to an object of the
new representation type.
:rep-type The representation type on which the translator
operates.
:lisp-to-rep-fn A lambda expression that accepts a lisp-type
value and returns a value whose type is the LISP
equivalent of rep-type.
:rep-to-lisp-fn A lambda expression that accepts a LISP value
that is equivalent to rep-type and returns a
lisp-type value.
For example, here is a possible definition of the selection
representation type:
(def-rep-type selection (size element-list)
(assert (<= 1 size 32))
`(translator-rep-type
:lisp-type (member ,element-list)
:rep-type (unsigned-integer ,size)
:lisp-to-rep-fn
(lambda (v)
(position v ',element-list))
:rep-to-lisp-fn
(lambda (n)
(nth n ',element-list))))
27.3 Foreign Structures
------- ----------
Foreign structures manage the data that LISP code shares with
foreign code. Any object that is to be accessible to LISP and to a
foreign routine must be created and accessed through the foreign
structure facility; this applies to the simple variable and to the
typical structure comprising multiple components.
The macro def-foreign-struct is a direct analog to defstruct.
When invoked, it generates routines to create and access instances
of a new structure type. Where its options are essentially the
same as those of defstruct, def-foreign-struct uses the same names.
def-foreign-struct name-and-options {doc-string} [Macro]
{slot-description}**+
The name of a new foreign structure must be a symbol; it becomes
the name of a new data type consisting of all instances of the
foreign structure. The name is returned as the value of
def-foreign-struct.
If the optional string doc-string is present, then it is
attached to the name as a documentation string of the type
structure; see documentation.
If no options are specified, then the symbol name may be
written in place of (name). Options are discussed below.
Each slot description has the following form:
(slot-name rep-type
slot-option-1 slot-option-value-1
slot-option-2 slot-option-value-2
...)
where the possible slot-options are :read-only and :initial-value.
Note that the type of the slot is not an option in
def-foreign-struct; each slot-name must be followed by its repre-
sentation type specifier.
The following example defines a foreign space ship:
(def-foreign-struct ship
(name (string 25) :read-only t)
(x-pos float :initial-value 0.0)
(y-pos float :initial-value 0.0))
This defines a new type, ship, whose elements are objects with
three fields. Evaluation of this example produces these functions:
make-ship A constructor function that takes three keyword
arguments; see defstruct.
copy-ship A copier function that takes a ship and returns a
bit-wise copy.
bind-ship A binder function; see "Foreign Structure Options,"
below.
ship-p A predicate that returns non-nil if its argument is a
ship.
ship-name Functions to access the three slots of a ship
ship-x-pos instance. Use setf to alter the slots ship-x-pos and
ship-y-pos ship-y-pos; ship-name cannot be altered because it is
specified :read-only.
def-foreign-struct also arranges for the printer to display a
foreign ship instance as:
#S(SHIP NAME <name> X-POS <x-pos> Y-POS <y-pos>)
and for the reader to accept this notation for a ship.
The size of an instance of any foreign structure type is
determined by the representation types of its fields. To learn the
size of a foreign structure, use the following function:
foreign-structure-size fstruct [Function]
This accepts a symbol that names a foreign structure and returns
its size in bits.
27.3.1 Foreign Structure Options
------- --------- -------
The def-foreign-struct macro accepts some of the options to
defstruct, and some that are unique to foreign structures:
:constructor
Use this option to rename or suppress the generation of a construc-
tor function, or to define multiple constructor functions; see
defstruct.
:copier
:print-function
:predicate
Use these options to rename or suppress the generation of the
respective functions; see defstruct.
:binder
Use this option to control the generation of a binder function.
This function accepts a foreign structure or a static variable
declared in foreign code, and produces an alias of that object; the
object can then be interpreted as a foreign structure of the new
type. If the argument is a static variable, its name is passed to
the binder function as a string. LISP builds an alias by looking
up the variable name in the symbol table of the foreign code
region.
As an example of the alias facility, assume that a variable
MOTHER←SHIP is defined in foreign code; it is a record whose first
three fields can be interpreted as a 25-character string and two
32-bit floating-point numbers. Using the binder function generated
in the space ship example above, the invocation
(bind-ship "MOTHER←SHIP")
will return an object that can be accessed, modified, printed, and
copied using the operations associated with the foreign structure
type ship.
The binder function is commonly used to alias an object that
must be passed or returned as any-pointer or any-foreign-structure.
The aliased object must not be smaller than the foreign structure
type used to interpret it. There is no way to protect the user
from reaching beyond the limits of the aliased object.
:addr-mode
:passing-mode
:returning-mode
:alignment
The default attributes of a new foreign structure type are
:addr-mode :word-aligned, :passing-mode :by-reference,
:returning-mode :in-ac2, and :alignment 16. Use these options to
override the defaults.
27.4 Foreign Entries
------- -------
Many Data General programming languages follow a common
language runtime calling protocol. This defines general rules
about the configuration of the stack and how to pass arguments. It
does not specify the details of how an individual language repre-
sents its own types, or which argument passing rules it uses. For
example, PL/I passes arguments by reference while the C programming
language passes by value. For more information, see the individual
language reference manuals.
To call a routine written in another language, LISP must know
the following:
1) its address,
2) the bit representation of its arguments and how the argu-
ments are passed, and
3) the representation and passing protocol of its return
value, if the routine is a function.
The user communicates all this information to LISP by invoking
the macro def-foreign-entry. def-foreign-entry takes the informa-
tion provided and produces a macro that is used to call the foreign
routine. def-foreign-entry generates a macro, rather than a
function, so that generalized variables can be passed as arguments
to routines that modify their arguments.
def-foreign-entry name-and-options {argument-spec}* [Macro]
This creates a macro that can call a procedure or function written
in a language other than COMMON LISP. name-and-options specifies a
name for the calling macro and its associated options in the form
(name option1 option2 ...). If no options are specified, only the
symbol name is required; this symbol is returned as the value of
def-foreign-entry.
By default, def-foreign-entry assumes that the symbol-name of
its name argument identifies the foreign routine to be called.
When the option
(:link-name string-or-symbol)
is present, it specifies a name for the foreign routine different
from the name of the foreign entry.
The option
(:returns rep-type)
indicates that the foreign routine is a function that returns a
value of the specified representation type; LISP infers the retur-
ning rules for that value from the attributes of the representation
type.
def-foreign-entry takes any number of argument specifications,
each of this form:
(argument-name rep-type {:direction dir})
argument-name is a symbol and rep-type is the representation type
of the argument. If :direction is specified, it must be one of
:in, :out, or :in-out; the default is :direction :in.
The following def-foreign-entry defines a foreign entry,
display-ship, that accepts two arguments: a foreign structure to be
displayed, and a boolean indicator that tells the routine whether
to clear the screen.
(def-foreign-entry (display-ship (:link-name "PRINT←SHIP"))
(the-ship (foreign-structure ship))
(clear-screen boolean))
When invoked, display-ship will call the foreign routine
PRINT←SHIP.
LISP retrieves the three kinds of information it needs to call
a routine in another language as follows:
1) It looks up the name of the routine in the symbol table of
the foreign code region to obtain the routine's address at
load time.
2) It derives the bit representation and passing rules of the
arguments from the representation types associated with the
arguments.
3) It derives the representation and passing protocol of any
returned value from the representation type specified in
the :returns option.